Papers with majority baseline
Identifying and Resolving Annotation Changes for Natural Language Understanding (2021.naacl-industry)
Copied to clipboard
| Challenge: | Annotation conflict resolution is crucial for machine learning, says a new study . past work on annotation conflict resolution assumed data is collected at once . a a supervised neural model can resolve conflicts in data annotation but requires access to high-quality data . |
| Approach: | They propose an approach to resolve annotation conflicts in a real-world context using a German dialog system. |
| Outcome: | The proposed approach improves on a real-world dataset with 3.5M utterances in German. |
A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applications (N18-1)
Copied to clipboard
Dongyeop Kang, Waleed Ammar, Bhavana Dalvi, Madeleine van Zuylen, Sebastian Kohlmeier, Eduard Hovy, Roy Schwartz
| Challenge: | a dataset of 14.7K paper drafts and accept/reject decisions in top-tier venues including ACL, NIPS and ICLR is presented to study peer reviews. |
| Approach: | They propose to use the dataset to collect peer reviews from top-tier venues including ACL, NIPS and ICLR and to use it to create a dataset of peer reviews for research purposes. |
| Outcome: | The proposed dataset includes 14.7K paper drafts and accept/reject decisions in top-tier venues including ACL, NIPS and ICLR. |
Disentangling Indirect Answers to Yes-No Questions in Real Conversations (2022.naacl-main)
Copied to clipboard
Krishna Sanagavarapu, Jathin Singaraju, Anusha Kakileti, Anirudh Kaza, Aaron Mathews, Helen Li, Nathan Brito, Eduardo Blanco
| Challenge: | Existing models with synthetic indirect answers to yes-no questions are not beneficial when working with real conversations. |
| Approach: | They propose to annotate the underlying direct answers to yes-no questions in real conversations. |
| Outcome: | The proposed model outperforms the majority baseline but the task remains a challenge. |
Discrete and Soft Prompting for Multilingual Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | In few-shot learning, discrete and soft prompting perform better than finetuning in multilingual cases. |
| Approach: | They show that discrete and soft prompting perform better than finetuning in crosslingual transfer and in-language training of multilingual natural language inference. |
| Outcome: | The proposed prompting model outperforms finetuning in crosslingual transfer and in-language training of multilingual natural language inference. |